Marginal productivity index policies for scheduling restless bandits with switching penalties
نویسنده
چکیده
We address the dynamic scheduling problem for discrete-state restless bandits, where sequence-independent setup penalties (costs or delays) are incurred when starting work on a project. We reformulate such problems as restless bandit problems without setup penalties, and then deploy the theory of marginal productivity indices (MPIs) and partial conservation laws (PCLs) we have introduced and developed in recent work, building on and extending previous work by Gittins (1979) and Whittle (1988). As a result, we obtain new dynamic index policies for scheduling restless bandits with setup penalties.
منابع مشابه
Restless Bandit Marginal Productivity Indices, Diminishing Returns, and Optimal Control of Make-to-Order/Make-to-Stock M/G/1 Queues
This paper presents a framework grounded on convex optimization and economics ideas to solve by index policies problems of optimal dynamic allocation of effort to a discrete-state (finite or countable) binary-action (work/rest) semi-Markov restless bandit project, elucidating issues raised by previous work. Its contributions include: (i) the concept of a restless bandit’s marginal productivity ...
متن کاملA Faster Index Algorithm and a Computational Study for Bandits with Switching Costs
We address the intractable multi-armed bandit problem with switching costs, for which Asawa and Teneketzis introduced in [M. Asawa and D. Teneketzis. 1996. Multi-armed bandits with switching penalties. IEEE Trans. Automat. Control, 41 328–348] an index that partially characterizes optimal policies, attaching to each project state a “continuation index” (its Gittins index) and a “switching index...
متن کاملMarginal productivity index policies for scheduling a multiclass delay-/loss-sensitive queue
This paper addresses the problem of scheduling a Markovian multiclass queue with a finite dedicated buffer for each class, where class-dependent linear holding and rejection cost rates model differing levels of tolerance to delay and loss. The goal is to design well-grounded and tractable scheduling policies that nearly minimize expected total discounted or long-run average cost. New dynamic in...
متن کاملOptimality of Index Policies for Stochastic Scheduling with Switching Penalties Author(s):
We investigate the impact of switching penalties on the nature of optimal scheduling policies for systems of parallel queues without arrivals. We study two types of switching penalties incurred when switching between queues: lump sum costs and time delays. Under the assumption that the service periods of jobs in a given queue possess the same distribution, we derive an index rule that defines a...
متن کاملDynamic priority allocation via restless bandit marginal productivity indices
This paper surveys recent work by the author on the theoretical and algorithmic aspects of restless bandit indexation as well as on its application to a variety of problems involving the dynamic allocation of priority to multiple stochastic projects. The main aim is to present ideas and methods in an accessible form that can be of use to researchers addressing problems of such a kind. Besides b...
متن کامل